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ABSTRACT 

Over  the  past  several  years ,  legged  mechanisms  have  shown  remarkable  progress  towards  practicality.  The  need  now  is  for 
efficient  means  of  controlling  these  mechanisms.  In  previous  papers,  we  have  found  the  optimal  switching  curve  for  controlling  a 
simplified  model  of  walking  called  the  rimless  wheel  model.  In  this  paper,  we  again  find  the  optimal  switching  curve  for  the 
rimless  wheel  using  another  approach— N elder  and  Mead’s  simplex  algorithm,  a  technique  for  global  optimization.  We  compare 
the  results  from  Nelder  Mead  with  those  obtained  from  our  previous  research  using  Pontryagin ’s  Maximum  Principle.  We  extend 
these  results  by  applying  Nelder  Mead  optimization  to  the  walking  gait  of  a  multiple  link  model.  We  find  an  initial  example  of 
efficient  control  using  this  powerful  technique. 


INTRODUCTION 

Legged  systems  have  always  held  a  promise  (albeit  a 
somewhat  futuristic  one)  of  mobility  [1].  Until  recently, 
most  research  in  legged  robotics  has  indeed  focused  on 
stability.  Recent  research  [2]  has  come  to  the  conclusion 
that  stability  is  a  necessary,  but  not  sufficient  condition. 
Efficiency  is  the  key  to  gaining  the  true  benefits  of  walking 
mechanisms.  Particularly,  we  examine  the  efficiency  of  the 
powering  scheme,  u(t),  of  a  legged  robot.  In  previous  papers 
[3-4]  we  have  shown  the  existence  of  an  optimal  switching 
curve  for  a  simplified  model  of  walking  based  on  Tad 
McGeer’s  rimless  wheel  model  [5].  In  this  paper,  we  find  an 
optimal  switching  curve  for  the  rimless  wheel  model  using 
Nelder  Mead  Optimization  and  compare  the  results  from  our 
previous  analysis  using  Pontryagin’ s  Maximum  Principle. 
We  extend  these  results  to  the  walking  gait  of  a  multiple  link 
model. 

NELDER  MEAD  OPTIMIZATION 

Nelder  and  Mead’s  simplex  algorithm  [6]  is  a  popular, 
time -proven  technique  for  optimizing  general  (nonsmooth) 
multivariable  functions.  Optimization  algorithms  of  this 
type  are  called  direct  methods.  They  are  robust  and  general 
and  applicable  to  the  problem  of  gait  determination  by 
parameterizing  the  gait  and  defining  a  metric  over  these 
parameters  for  optimization. 

For  function  f(-)  defined  on  an  ^-dimensional  domain 
space,  Nelder  and  Mead’s  method  finds  a  minimum  using  a 


simplex  formed  with  n+ 1  points.  The  process  of  finding  a 
minimum  proceeds  through  a  process  of  transforming  the 
simplex  by  1)  shifting  points  in  the  direction  of  a  minimum, 
2)  shrinking,  and  3)  expanding.  In  addition,  a  restarting 
criterion  can  be  used.  This  restarting  criterion  uses  the 
concept  of  a  simplex  gradient.  The  Nelder-Mead  algorithm 
continually  updates  a  simplex  in  the  TV-dimensional  space  of 
the  problem.  For  N=3,  this  simplex  is  a  tetrahedron  as 
illustrated  figure  1.  There  are  TV+1  vertices  and  TV+ 1 
corresponding  function  values. 


N- Dimensional  Simplex 


Figure  1:  Nelder-Mead  uses  function  evaluation  over  the 
vertices  of  an  TV-dimensional  simplex  to  optimize  a  criterion 
function  over  an  TV-dimensional  space.  Shown  here  is  a 
tetrahedron,  which  is  a  simplex  for  TV=3.  There  are  N+l 
vertices  in  an  TV-dimensional  simplex. 
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Nelder  Mead  Steps 


Order  the  points  from  lowest  cost  to  greatest 
Calculate  the  values  of  each  simplex  point 
Try  new  point  to  replace  the  worst  point.  The 
algorithm  attempts  each  step  below  in  order  until  the 
cost  of  the  proposed  point  is  less  than  that  of  the 
worst  point. 

Reflection  -  reflects  worst  point  about  the  center 
of  the  simplex 

Expansion  -  expands  in  the  direction  of  the  best 
point 

Contraction  -  shifts  worst  point  towards  the  best 
point 

Reduction  -  if  no  other  options  are  available, 
each  simplex  point  except  the  best  is  shrunk  towards 
the  center 

Accept  the  point  once  a  step  successfully  reduces 
the  cost 


Repeat  process  until  a  pre-specified  tolerance  is 
met 


APPROXIMATION  OF  THE  SWITCHING  CURVE 
FOR  THE  SIMPLE  PENDULUM 


Figure  2:  The  rimless  wheel  model  of  walking  acts  like  a 
powered  simple  pendulum  (between  foot  collisions). 

We  can  describe  the  cost  of  powering  the  rimless  wheel  as 
a  combination  of  energy  and  time  cost.  We  designate  the 
cost,  J,  composed  of  a  time  cost,  T,  with  a  proportionality 
constant,  k  and  an  energy  cost,  E  for  a  given  function  of 
torque,  u(t).  The  “optimal”  control  function  u*(t)  thus 
requires  little  energy  but  still  reaches  the  destination  quickly. 


7r+a 

7=|  ud0  +  kT  (1) 

7i— a 


Equation  1  describes  a.  functional,  i.e.,  J  is  a  function  of  a 
function.  Since  u(t)  could  take  on  any  shape,  we  describe  it 
using  a  high  degree  of  freedom  system  (20-40  components). 
These  functional  components  were  optimized  using  Nelder- 
Mead  in  order  to  minimize  the  cost  functional,  J.  As  we  saw 
in  previous  results  using  Pontryagin’s  Maximum  Principle, 
when  we  optimized  the  torque  function,  we  found  evidence 
of  a  “switching  curve.”  However,  the  result  shown  in  figure 
3  was  only  found  after  adjusting  the  initial  simplex 
parameters  and  adding  a  sufficient  number  of  components. 
We  used  Nelder-Mead  to  optimize  the  coefficients  of  a 
Fourier  series  (half-range  expansion  of  the  torque  function) 
to  approximate  the  solution. 
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Figure  3:  Fourier  Series  half  range  expansion  found  using 
Nelder-Mead  Optimization 

Likewise,  we  performed  a  Nelder-Mead  optimization  on 
the  cubic  spline  components  of  the  torque  function. 
Pontryagin’s  Maximum  Principle  provides  the  correct 
answer  to  this  problem  at  k  =  5.  The  solution  is  a  switching 
curve  at  0.6s  with  a  cost  of  5.34.  Nelder-Mead  confirms  this 
result. 


Figure  4:  Spline  Components  found  in  Nelder-Mead 
Optimization 
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Note  on  Nelder  Mead 

Due  to  the  many  local  minima,  we  found  that  it  was 
critical  to  set  the  maximum  value  of  torque,  u(t)  <  1  for  the 
simple  pendulum.  Without  limiting  the  results  to  the 
maximum  torque,  Nelder-Mead  would  shoot  briefly  shoot 
above  the  maximum,  in  essence  concluding  that  the  square 
error  we  set  was  worth  the  penalty.  To  achieve  a  soft  sort  of 
clipping,  we  used  a  sine  function  to  obtain  the  results  shown. 
After  Nelder  Mead  obtained  a  spline  function  for  u(t ),  we 
would  feed  Sin [u(t)]  into  the  cost  function.  If  we  were  to 
hard  clip  u(t)  at  the  maximum,  this  might  feed  sharp  corners 
into  our  Runge-Kutta  integrator,  like  sticking  a  fork  into  the 
garbage  disposal. 

MULTIPLE  LINK  ROBOT  CONTROL 

To  apply  this  approach  to  the  gait  optimization  problem,  a 
two-link  pendulum  model  was  constructed,  as  illustrated 
below.  It  has  two  rotational  degrees  of  freedom. 


{x,y} 


0i 


Figure  5:  Multiple  Link  Robot  Model  used  for  illustrating 
optimized  stride  construction. 


Each  link  length  is  1  m.  The  masses  of  the  links  are  at  the 
ends  of  the  links,  and  the  mass  values  are  variable — 

generally,  ml  >  m2 .  For  optimizing  the  stride,  the  point  in 

the  lower  right  about  which  6X  rotates  is  assumed  fixed. 

The  location  of  the  point  at  the  end  of  the  second  link  is 
represented  in  inertial  coordinates  as  {x,  y}.  To 
parameterize  a  stride,  the  torque  input  on  the  second  joint 

( d2  in  the  figure  above)  was  constructed  from  a  length- 
p  +  3  vector  of  real  numbers: 


*1 

T1 

*2 

X 

T 

p 

P 

X  , 

T 

p+ 1 

•X  „ 

4 

p+2 

A. 

(2) 


In  (2),  the  last  two  numbers  provide  the  initial  conditions 
on  the  joint  rates.  The  first  p  numbers  are  torques  and  form 
the  domain  values  of  knots  for  a  spline,  typically — but  not 
necessarily — a  cubic  spline.  Splines  give  a  continuous  and 
continuously  differentiable  function  that  is  well  behaved  and 
practical  to  implement.  These  values  are  equally  distributed 
over  a  time  determined  by  the  number  in  X  at  the  p  + 1 
position.  Let  this  curve  be  defined  as  P[X].  The  graphic 
below  illustrates  its  formation. 


Figure  6:  Illustrations  of  torque  input  functions  P[X] 
constructed  using  the  defined  parameterization.  The 

parameters  6X  and  02  do  not  affect  the  shape  of  the  curve. 
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Let  T  be  the  column  vector  of  torques  on  the  joints,  0  be 
the  column  vector  of  joint  values  (i.e.,  0  =  \6]  02f )  ,  M(0) 

be  the  inertia  matrix,  C(0, 0)  be  the  vector  of  Coriolis  and 
centripetal  terms,  and  G(0)  be  the  vector  of  gravitational 
torques.  With  these  definitions,  the  system  dynamics  for  the 
model  shown  in  5  are  given  through  the  following: 

t  =  M0  +  C(0, 0)  +  G(0)  (3) 

We  do  not  consider  frictional  forces  in  this  study.  To  form 
a  metric  for  optimization,  we  set 


P(X) 

0 


(4) 


We  integrate  the  differential  equation  in  (3)  over  the  time 
period  of  P(X)  using  the  last  two  parameters  in  X  as  the 
initial  joint  rates.  The  initial  joint  positions  are  fixed  in  this 
study.  A  metric  H  for  optimization  is  formed  by  combining 

several  submetrics  h{  using  positive  weights  W-  as  follows: 


«=2>a 


(5) 


T  - - 

H  =  j{p[x]}2  dt  +  8000^(x-xd)2  +(y-yd)2  +... 

0 

. . . + 2oo(4tm4  -  d/mef  )2  (6> 

This  is  meant  to  be  one  example  of  a  selection  that  could 
be  used.  With  the  high  weights  used,  it  effectively  seeks  to 
drive  the  point-placement  error  and  the  energy  difference  to 
zero  while  reducing  the  magnitude  of  the  applied  torque. 

This  configuration  and  metric  selection  was  used  with 
randomized  Nelder-Mead  optimization  to  calculate  a  stride 
by  calculating  20  results  and  choosing  the  one  with  the 
smallest  metric  value.  Based  on  using  four  torque  values  to 
build  the  spline  curve,  the  following  value  was  calculated: 


"  -0.931366" 

T2 

-0.146139 

0.245392 

= 

-0.355268 

T 

1.07978 

d 

1.97858 

A. 

-1.10653  _ 

For  our  studies,  the  submetrics  were  selected  from  among 
the  following: 


These  give  a  metric  value  of  0.142565,  and  produce  the 
following  very  low  input  torque: 


The  integral  of  the  input  torque  squared: 

hi=\{p[x](t)}2  dt 

o 

The  Euclidean  error  in  end-point  placement: 

4  =^/(x-^)2+(y-^)2 

A  measure  of  the  difference  in  kinetic  energy 

ht  =[9?M9i-9fTM9f)2 

With  this  choice  of  metrics,  we  were  able  to  calculate 
strides  by  optimizing  X. 

Example 

For  an  example,  the  masses  are  set  to  ml  =  /5  and  m2  =  1 . 
Gravity  is  9.8  m/s2  along  vertical.  The  initial  values  for  the 
joints  are  0X  =  5k/6  radians  and  02  =-27/i  radians.  The 
metric  is  set  to  the  following: 


Input  Torque 


This  torque,  when  input  to  the  system  defined  through  (3) 
and  integrated  gives  the  following  output  which  is  illustrated 
through  a  sequential  rendering  of  the  system  in  figure  8. 
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Figure  8:  Screen  shots  from  the  optimized  stride. 

The  stride  produced  by  the  input  torque  selected  is  shown  in 
the  figure  above.  The  endpoint  ends  approximately 
symmetrically  offset  from  its  starting  position  and  the 
system  ends  with  approximately  the  kinetic  energy  with 
which  it  started. 


CONCLUSIONS  AND  FUTURE  WORK 

The  switching  curve  we  have  shown  for  the  simple 
pendulum  agrees  closely  with  the  global  optimal  control 
u*(t)  obtained  through  Pontryagin’s  Maximum  Principle 
(PMP)  and  other  methods.  The  result  we  show  for  the 
double  pendulum  should  be  interpreted  not  as  a  claim  of 
global  optimal  control,  but  as  a  typical  example  of  an 
optimized  stride  for  a  more  realistic  walking  model. 

The  results  we  found  were  not  obtained  by  assuming  any 
type  of  switching  curve,  i.e.,  we  did  not  feed  Nelder  Mead 
the  answer;  however,  we  knew  a  priori  that  the  result  (at 
least  for  the  simple  pendulum)  should  look  like  a  switching 
curve.  Perhaps  we  found  what  we  were  looking  for,  because 
the  results  we  show  in  figures  3  and  4  were  not  typical.  In 
fact,  you  could  find  many  local  minima  for  u(t)  that  looked 
nothing  like  a  switching  curve  but  had  a  cost  within  ten 
percent  or  so  of  the  optimal  cost.  Without  the  knowledge 
that  a  switching  curve  was  the  optimal  control,  we  could 
have  easily  given  up  the  search  for  the  global  minimum. 
Like  traveling  with  a  map,  we  found  the  previous  results 
were  a  guide  to  Nelder  Mead. 

For  the  double  pendulum  we  see  an  example  of  an 
optimized  stride.  Is  this  a  global  optimum?  We  simply  do 
not  know.  We  would  like  to  further  research  these  results 
using  PMP  or  a  similar  technique.  It  may  be  that  strict 
optimality  is  not  critical,  but  rather  knowing  that  you  are 
close. 


The  nice  thing  about  having  a  stationary  switching  curve 
in  phase  space  is  that  you  have  a  simple  “map”  to  follow  no 
matter  where  you  find  yourself.  Thus,  if  you  deviated  from 
the  optimal  path,  you  could  simply  recalculate  your  position 
on  the  “map”  and  shoot  for  switching  curve.  Contrast  that 
with  trying  to  follow  an  optimal  stride  without  this 
information.  If  you  deviate  too  far  from  the  optimal  path, 
you  might  not  be  able  to  find  your  way  back.  That  being 
said,  for  repetitive  tasks  like  walking,  it  is  likely  desirable  to 
strictly  follow  a  known  optimal  stride. 
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